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ABSTRACT 

Problems in evaluating parent- teacher. interaction are 
examin2d. Fictional exasples. are used to describe three main types of 
problems: 1) the duration and frequency of evaluation, especially 
after training or conferencing has ceased; 2) alteration of 
measurement instruments and procedures; and 3) methodological ani 
research design problems. Suggested are procedures such as 
standardizing measures over the duration of the treatment and 
followup, and the use of a control arene to strengthen the research 
design. (CL) ; 
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FVALUATION PROCEDURES IN PARENT-TEACHFR CONFERENCING 
That beneficial effects accrue from parent teacher in- 


teractions is the general assumption that underlies a wide 7 : 


variety of activities ranging from sendinp-home flyers 

about school activities, notes and prorress rdports relative , 
to a child' s academic and social prorress, to extensive eit 
and/or elaborate systems of parental involvement in parent 
activities within the classroom, at home ,. and in a variety 


’ 4 
of roles and canhetiles. However, if one accepts the asser- 


tion by Kroth and Simpson (in arene) hae dhe effects of . 
papentes. tenchens, Sc enawlans and therapists — he for 
better or worse, and that inadequate resources: (skills, 
knowledge) ei the part of the helper, and an inappropriate 
annoees: can lead to a aeteciGnandongin performance on the 
part of the person being "helped', as well as improved ; 
_ functioning." (de. 194), it would appear that concern re- 
garding the efficacy of parent-teacher conferencing and 
parent training is leritimate and warranted... ) 
That parents can be taurht to improve their children's 
behavior is obvious from recent reviews by Berkowitz and 
> Graziano (1972), Johnson and Katz (1973), and 0'Del1 (1974). 
There are also indications that exceptional children 


benefit from parental involvement in the educative process 


(Alexander, 1977; Clements .& Alexander, 1975; Feldman, 
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Byalick & Rosedale, 1975; rua & neloach, 1975;. 
Kroth, Whelan & Stables 1970; Tavormina, 1975). * 
There are, however, numerous questions: which remain 
~ unanswered in retation to parent involvement, conferencing, 
and training. A Aaine? of ‘these’ .concerns have heen de- 
lineated by the‘ reviewers and ‘authors: cited ‘above. Some 
concerns may have Rreater or lesser interest or significance, 
‘depending on one’ Ss recite or function in working with parents, 
whereas .other concerns -have implications that are pervasive 
across ‘the field. While concerns or problems of evaluation 
are often ‘interrelated and interact, an attempt will be made 


to identify a number of these separately, through 
fictional examples 1 and to surrest vrocedures: that will 
. b] . 


foster. improvement in évaluation of parent-teacher inter- 


= actions. 


Problem 1: Duration dnd frequency of evaluation. 


A repeatedly cited (Forehand & Atkenson, in press; 
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Johnson & Katz, 1973; O'Dell, 1974; Bernal % Margolin, Note 


1) and Reieer ve concern in assessing the efficacy of 
-, parent iendtnine and ontereneine has been _the duration and. 
frequency of measurement, especially after =e or 
conferencing has ceased. Often, most interventions are of 
a short ieuition by intent or nedsastay, and follow-up" 
evaluation is also Pt limited duration and/or frequency ® ° 
| 4° ; 
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with the rationale most often presumed to be expediency, ‘ 
economy and/or difficulties in obtaining measures once 


‘training or conferencing has ceased. 


% 


Insert Figure 1, la about here _* 


—_—K—X—X—X—nKx——SSgyre.9.V—.VXVX._._._.]]]____—_eueuvuxcx_c_c_cxcxo“=— 


- 


. Example 1 | | | . 


An example to illustrate this problem is the teacher or 
therapist wie ania nepents to collect measures on mutip ls 
behaviors several times per day or all day, do reliability 
checks, graph the. data daily, perhaps usifg thig'as a | 
pelepien for admittance to the next training session, 
utilizing the records for decisions regarding ‘programming, 
consequatién, etc. The trainer then comptes intervention 
@Basures to baseline, decides that the parents have 
"learned" as demonstrated by the changes in the observed 
abies of the child, and tells them to come back in a 
month (or 3 months, 6 months, etc.) and perhaps bring with . 

them a record of the child's behavior for several days 

preceding the meeting. | 
An assumption is that the parents will ‘continue the 
program at the same diiron bx levél of intensity even . a 
4 though the environmental conditions and contingencies which 


are assumed to have oggasioned and maintained ‘the parents' 
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behavioral change ‘have been drastically. altered; that the 
most recent record of the child's behavior adequately: \ 
Fepresents Be or her behavior over ‘the period of time 

from termination \to post-check, and that, it is predictive 


X 


of his or howe satire behavior.» % | ‘ 


we * 
of foliowaup is the, seemingly arbitrary selection of 


2 ‘One other problem a) concern. in. relation to frequency 
points in time at eae ! llow-up measures are to be < 
obtained. . The dates \er imes sn tate seem to bear, lane 

or no relationship Ae type, rate, duration. or probable 
durability of the beh vi rs of concern. Whereas’ ‘some \types \ 
or clagses of behavior |a e quite variable, others are more | 
" stable. Investigators } uld be well advised to note those 
variables waite ihtensiy measurement procedures are being 
utilized, wad to Yan appropriate and differential fre- | 
quencies and durations of follow-up measures to take into 


‘account those dissimilarities. 


Problem 2: Alteration of measurement instruments and pro- 


cedures. - 
| * 


A problem that often appears conturrently with the 


first is a.change in measurement procedures; including type 


toe 


of measurement utilized, relative precision of observations, af 


accuracy of recording and/or response cost in reporting 


procedures. It appears\that these problems ‘frequently occur 
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t,a most critical point in time; at the termination of 


f 


_ training or toward the conclusion of an extended sequence 


of parent confererices, and are obvious at follow-up, with | 
4 \ 


probes, or. post-check evaluation. This is illustrated in ~ 


a 


the example below. 


. 
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‘Insert Figure 2,2a, 2b about here 
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Example 2 


During an extended series of conferences, parents | 
were asked to observe and record daily the frequency of 


requests and commands they made or gave their child, the 


-frequency of his compliance, the: elapsed time from when 


the. command or request was delivered to when the child 


-— , 


initiated-compliance, and how long it took him to complete 


the task. They were also instructed to record this infor- 


"mation simultaneously at least once per week to obtain a 


measure of interobserver agreement. After baseline was 
obtained, they were instructed to place the child in a 
time-out area of the house (the foyer) for 5 minutes on 


the child did not initiate compliance 


each occasion when 
° q \ : : ‘ 


with 15 seconds. 


‘The parents were requested to graph this data daily, 


{ 


and bring all records to a week]y conference for six weeks. 


- An ABAB reversal design (Hall, 1971) was utilized to verify 


~< : 
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effects. A decrease in noncompliant behavior of 30 per cent 
or more was used as_a criterion for success. (Eyberg & 


Johnson, .1974). The parents were then urged to continue the 


fo 


procedures as necessary. — | , 

Two weeks later, the mother was contacted by telephone, 
asked shout frequency of compliance, frequsies'at use of - 
‘the time-out procedures, the use of the procedure in 
“gelation to any other. problem behaviors of the child, and 
two things were going generally. She was also encouraged 
to call if any problems arose. One month after termina- | 
tion, -a questionnaire was mailed to the parents requesting 
that the parents rate on a Likert-type scale (Brown, 1976) 
their perceptions of the effectiveness ofthe training, the 
degree of change effected, the various components of the 

Teena and generalization of the use of the procedures. 

A week eubaeguent to the nada ine: the mother was contacteg .. ‘ 
by telephone and”urged to complete. and return the ee | 
“naire. Three months aibeasuant to termination, the mother 
was again contacted by telephone. She indicated that 
things were going fine, and the behavior was no longer a 
eecbion. Four months subsequent to termination, an | 
attempt was made to contact the mother by ‘phone. The 
operator indicated that the number had been changed at 


the owner's request, and was now unlisted. The treatment 
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was judged to have been a success. 

Problems in this éximple include alterations of the 
‘frequency of measurement, a change in “the type of measure- 
ment information requested, no information requested about 
some of the eeumives previously obtained regularly, | 
| requests for new pdteded Gish about previously unreported. 
‘behaviors, and a request for a ‘global subjective assessment. 
In addition, a new measure was introduced, as well as new 
‘and ‘different response demands 16x the parents' behavior 
in the method used torobtain reports about the behavior.: 

J ie cites appears ike x tacit agreement or: under- 
standing“ takes, place toward or at the end of training or 
conferencing hakuasn the teacher or therapist and the 
parents; that they now. "know the ropes" as demonstrated by 
their extensive recordkeeping, graphing and consequation 
-over the past several weeks, and that they will continue to” 
apply (and perhaps expand on) the procedures and principles ° 
without the bothersome necessity of ‘continued measurement 
of behavior. The teacher/trainer also seems, to reduce 
expectancies and criterions of acceptable levels of : 
measurement over time and as contact with the parents — ; 
comes less frequent. Whereas in the first several weeks, 
parents‘ wefe pannicwd: taught, urged, and reminded of the 


necessity and rationale for accurately measuring behavior 


("you must first determine tHe operant level; it's necessary . 


po 9” 


> 
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go ‘we'll know if the change procedure ig working; we must ~~ 
make . sure that the change is due to the trentnent and not 
to measurement arenes we need it to make judgements about 
* future procedures; we need it to determine. enéterion a 
« levels and goals; we need it to make probability whacomeits 
* about the behavior; etc.), in latter stages of conferencing - 
or training. and at follow-up, the teacher/trainer finds 
less rigorous and/or frequent meastires aeceptabie or- 

9 desirable. This seems incompatable with. the neincthien 
gene procedures of accurate measurement and evaluation, if 
the’ investigator is willing .to rely on less sequent and 
less rigorous measures interspersed over longer durations 

and in environments - probably less susceptable to his 
direction or stimulus. control, he should be at least as 
willing to accept the same ‘levels of measurement while : 
the parent training is under his immediate supervision. and 
CLSET AGES. - ; e ’ 
‘i A more appropriate expectancy would ‘be the necessity 
for more rigorous and frequent, measures as training-ts 
terminated; or at least measurement as adequate as during 
Co monitoring, feedback, and instruction is 
ECUFETER between the parents and instructor on a frequent, 


regular basis. If ongoing or follow-ap evaluation is an 


expectancy of the instructor, parents should be informed 


early in the training process, so that a rationale may be 


10 


‘ 


of the investi tien, including follow-up. This should jn- ; 
clude usi measures and procedures planned for follow-up 

the course of training or conferencing in conjunction 

#ith more rigorous or intensive measures. “This woutd allow . 
comparison of SRACEUMENTS for estimates of reliability ‘and 


- * 


validity, as well as providing at least some consietency 


of measurement over time throughout tntarvention and follow-. | at 
up. Attention should also be directed to the response’ 
- demands of various measures, instruments and procedures for 
the collection and reporting of information for PUEDORE® 

of etticicney end effectiveness (Cone & Sloop, 1973; | S 
Forehand & Atkeson, in press; Mash & Terdal, Note 2). Just | 
as the most precise instruments are of little value if 

they cannot be implemented by the investigator or are not 

‘ased by parents, instrumenta and measures which are % 
economical and apaidy cnéetned are of little or no value | 

if they are not reliable and valid measures of the be- 
haviors of concern. | Measures that have been demonstrated 


to produce desirable ratios of effectiveness/efficiency 


in relation to particular training procedures, response 
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classes or behaviors should ‘be used more ‘frequently, with 


adaptations as needed, rather than starting from "scratch" 
with each new conference, parent, or procedure. “The axiom 
that more is better in regard to measurement may be 
fallacious when the above considerations are taken into 
account. Further,. information as to why parents “drop 
out", fail to respond and are "lost" prior and subsequent 
to follow-up is sorely needed. : Boen information qpuld be 
, valuable in planning future fratuing;. to provide for 
additional Aiaantercanae training (Patterson, Teta) ¢ and 


potential alterations. in jnnteumention. 


£ 


Problem 3: Methodological and research design problems. «| 
‘While the previous vignettes (examples 1_and 2 above) Lek 


could-be considered as hampered with design op, methodology 
problens, their inclusion was not to demonstrate those 


concerns. Two short examples will serve as illustrations of- 


err 
ae, oe ‘some of the difficulties encountered in this area. — ‘ 
t, © i. . 
ee 
-Insert Figure 3, 3a, 3b about here 
Example 3 : 
A teacher decides to act on the assumption that parent 
conferencing will improve the behavior and attitude of the / 
$ . children he fenches: Consequently, he sends home notices bi ark - ke 
that a parent group is being formed, and invites parents : 
ae 2 
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Noo : eo 
to participate. One-third of the parents bacnenn affir- 
matively, and the parent group begins. poer the eight week. 


duration of the conferencing, some of the parents attend 


“sporadically, and others drop out. Nevertheless, the 
behavior ‘and attitude of the participants' children signi- 
ficantly improves from pre-to-post assessment. The teachar 
conclydes that this was an effective procedure. 

A number of shortcomings ‘in this example are phvioua: 


' including lack of controls for maturation, potential 


selective variables operating relative to attendence and 
participation; outcome, relative performance of parents er 

and children, etc. ‘However, these problems on,concerns 

may become less obvious, obscured, or ‘ignored when single Pa 
subject, {Sidman, 1960; Hersen & Barlow, 1976) or applied | a 
) behavior. “analysis (Hall, °1971) research designs are . beg am 
" superimposed on this or similar examples. It is apparent. 

that a control group would strengthen this design; what 

would constitute adequate controls is still open ta | 

question. If a detector control group (Levitt, 1963) or 

_ matched sample (Shepperd, Oppenheim, & Mitchell, 1966) “is A 
emp leyedy yeh Samia seem that the necessity for ‘accounting. 

for potentially ecserracent variables in. the groups would _ a? 

be at least twica’ as great, since dittivenoss of concern | 
would include not, only those inherent in the parents and M | os 


children, but also a number of pokentlnt interactions 


‘4 . * 
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between the two. The necessity for assurance of homo- 
geneity of variance is at issue in this instance, | and as 
violation of this assumption. will severely delimit con- 


‘widetona in regard to effectiveness of procedures.’ 


* \ , ‘ 


c 
, 


Insert Figure 4, 4a about here 


| Example 4 

An iuvaaeinater is interested in the question of 
whether behavioral parent training is more effective than 
. nondirective eauieoline in helping parents manage behavior 
. problems exhibited by their. children. Parents of children 
who attend one of three Behavior Disorders classrooms in 
an elementary, middle school, and a “high | school in a 
,Aocel wchool istrict s are rensonty assigned to one of the. 
two experimental sontitions or a control froup. Each 
training/counseling group is conducted by an individual 
who is competent and experienced in the treatment or 
training modality utilized. A number of valid, reliable 
measures are- used to evaluate children’ Ss aud parents" 
behavior prior, during, and, ‘Subsequent: to intervention. 
Results indicate that while both groups show improvement 
over the controls, there are no significant differences 
between the two. A conclusion that might be erroneously 


anauned te: ‘that while both ‘types: of intervention are ‘ef. 


ae L& et 
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fective, neither condition is more effective than the 
other in helping savents manage behavior problems. exhibited 
by their children. , 1 
The major problem inherent in this een ae the pos- 
sibility that individual and significant differences of. 
o responsiveness to one or the other treatment condition by 
a Wew or particular types of paente,: children. or behavior : 
problems may be "washed out" by the preponderant nonrespon- 
siveness of other typed of parents, children and/or behaviors 
nea Taree training or treatment modality. The assump- 
Lia of equality of responsiveness of any type of disorder, 
child or parent to any given type of training or treatment 
is probably, naive, as is the assumption that various t: 
rates, intensities and types of behavior disorders are 
comparable within or between spew and/or grade levels. 
A more productive approach might be the matching and 
assignment by type, rate, intensity or duration of the 
disorder to one of the three conditions; a preassessment 
matching and assignment of parents on variables that appear 
to have face validity as a basis for por@stial variability 
of responsiveness to training, i.e., matching and assign- — 
«ment of high, moderate, and low responsive or receptive | 
parents eanaliy 40° ‘conditions: and/or perhaps matching 
and assignmént across ate more classes of variables; | " 


such as parental attitude, level of severity of the child's 


« ‘ be ° ¥ F 4 
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behavior, and types of intervention. 

| Although the above may appear esoteric om undeananty: 
until we begin to account for obvious sources of error, 
less obvious and unknown arliables of parent conferencing 
£ koeenrere are unlikely to brought to light and accounted 
for, while the value and efficacy of these endeavors will 


continue to remain presumptive rather than demonstrable. 
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Footnotes 
JA 


‘these examples were/ fabricated for purposes of describ- 
ing problems of neamicenen’ and evaluation. They are 
not intended torepresent actual experinents, published 
or unpublished. Therefore, the data as. represented in 
figures 1-4 does not exist. ©! 1. 

Correspondence dang the articid shout be sent 
to Ronnie N. Alexander, College of: Multidi ciplinary 
Studies, The University of Texas a San Antonio, San tate: 
Texas 78285. a iy 
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; Figures.1, 1a. Examples of concer#s gas Oana 
na £ ; 


and frequency of ‘measurement. oo 


Figures 2, 2a, 2b. Examples of piteration of 


Y measurement and measurement procedures. 


Figures 3, 3a, 3b. Examples of research design and 
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problems of controls. oa 
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Figures 4, 4a. Examples of potential interactions of 
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variables. oe 


4 


. ¥ Lod 
" 
. Low 
‘ j ° \ ed 
cs 
: \ “a eo ; 
, = NN ~ 
bet i 
i 
~ S 
: “SR: 
988 ° * 
ry eS 
e. a - 
+ 
ay ~ 
ae wn > 
tre 
LO, + 
ty * i 
aN wn 
igi +. bald 
#3 2 
. foe) 
~) < wo e 
é F « ‘ 
é ° OS ee. Sa ae a ee, a ee Eg - a en em Ra 
~ - 
Q 4 
r 
: 4 
fa rm 
<¢ s : 
e * 
o 
N 
8 
a 
’ es 
| 
q i? 
> 
- ££ 
wo a 
- s 
* 
* 
Oo 
L med b 
" 
4 
“Yo 
a * 
<= 4 
Ps bad = 
‘+ o@ 
w Jo = 
: "vy oc = 
~~ co 
"ne 2a 
vor 
J a ‘ 
2 = 
o ag ae 
eoe : 


Frequency 


O Requests 
e Compliance | 
X Reliability checks 


2 


DAYS 


- 


“” 


* 


« 


a 


eo 


et Se FI re oe EK TO 


we eT a ee SN Be 
i . 3 7 


? 


. 


+ 


DAYS 


O Requests 


1 aaa 


aAoanbads : 


he 


 eptgremee igre 


eee | 
‘ 
' 


peeeermmee one 


Compliance 


wee ee 


Reliability checks 


X 


| 


‘ 


MINUTES 


a —_ 


60 


20. 


oO 


=_ ap omwep sw ew oe Se em sw ew ese om owe oh ace. 
= am an oe ob a on 


-—_p ap ae eae ow oe ae oe oP owe ow ow OB =P ow o= 


aS tf 
en ep a= ep am 2 a= == 


fl 


a Total aan of minutes to initiate tasks — 
9 Total -nunber of. minutes to complete. tasks 


some Giedemeetoetn 2 reaionaninrtrk so 2B ron ibchienieenipihieeniytihs 


t 


‘3 
Fo 
2 
a. | 

= 


. ¢ 
. 
zs . 
' a ; 


1. Fr@guency of coimands 
42. Frequency of compliance | 
3. Elasped time between command — 


' and initiation of action 


4, Elasped time for initiation 
to completion of tasks 


5. Reliability measures — 


* 


I ear iee a2 


AGE ee ae EE 


11: No “Pye oe EE, 1 2 ' 
‘12. Same + ¢ V2. No - 12. No | [ 
13. No 13. No 13.No ot. ' 
| I" by 3% | 4 
44. No 14. No apa. No 4 ' .: 
ie No ; iB No" ig No ) 1 
6. (New) Frequency . 6. No . No } 
“of use of time | ‘ | a, | 
out 1 | a | : . 
\7- (New) Generaliza- 7. Yes 17+ No | . 
- _ tions | . 
8. (New) How's‘it .8. No Yes - ; 
{° go me 1. 1 | i 
{ 19. New Response} 4 [* 
’ Demands . 4 
oe ci oo 
| Pea 
| vd q 
26.30 34. 38. 42 46 50 54 58. 62 


ee ceainhintiicantaniiekaaieeiaminanaian dank Sane reign 


30> 


* ; e v 23 
Py my 
' ‘ . L ; 
| 
| . 
. 
\ | ; 4 
* 
‘ 
¢ 
XQ 
i : = ae : . 2 St 
eg ' WEEKS Se 
Experimentals: @ Disruptive Behaviors 
- +  & Positive self referents a ‘0 ; 
aperents 3 L oe “otheetee ; Bh (epeteneybonraaltt ” S ey ” rg! ae soennamianl 32 
é i t : ‘ . 5 ; 
1 $ 


| ~ 34 
~- Controls: -O Disruptive Behaviors ; ; . ate | one . eoniescoaraeetins 
“dp Positive Self Referents iy eo , : 


oat | 


36 


——_— om ome eee ome ee ee ie oe ee ee ee ok ae oo 


Phd 
« 


forte ee 


aniline 


Po ad 


pga | 
Tn 


J 
. 
—_ = ee —— — — —- —=_ — — —_ 422 es eel come ee 
, 
~ - ‘ 
' 
: 
e 
j 
e oe 
a 
* 
4 
= =—=—- — = oo — — — Se =< — —_ me <a os — 
_—- 
. 
. ; 
Ly 
t 
: 
* t 
. i 
t 
. 
* : | 
4 
= =- a —_— —_ oa —_ ean ee «a= oa —_ => = —, ‘ 
‘ te 7 o 
” Py 


Multiple Measures 


i. 
$ Behavioral Counseling 
O-Non Directive Counseling 
A Gontrol Group 


PostTest 


PreTest 


~ PreTest 


PostTest * Pretest : PostTest 


a 


* 


Measures of Concern 


(Parents; and Children) 


low long _. moderate . aggressive high older 
cooperative * duration O cooperative behaviors _ cooperative children, 


~~ Moderate long A hig gh aggressive hig gh . ~ ” eTementar 
Ocooperative duration cooperative _ behaviors cooperative children 


Abigh long +a aggressive hi gh - younger 
cooperative aducetion cooperative behaviors A: cooperative ‘children 
. . ’ ° 


i he 


